feat(fuzz): enhance corpus mutation with all-call strategy and msg.value support#13177
feat(fuzz): enhance corpus mutation with all-call strategy and msg.value support#13177grandizzy wants to merge 8 commits intofoundry-rs:masterfrom
Conversation
f9e780c to
ea0af53
Compare
ea0af53 to
622c18a
Compare
…lue support When using the ABI mutation type in coverage-guided fuzzing: - 30% chance to mutate ALL calls in the sequence rather than just one - Mutate sender (15%) using addresses from dictionary - Mutate msg.value (15%) for payable functions Also adds automatic msg.value generation for payable functions during initial call generation, with value shown in sequence output. Value generation is biased towards smaller values to avoid balance issues: - 85% no value, 10% small (0-1000 wei), 4% medium (0.001 ETH), 1% large (1 ETH) Based on foundry-rs#8644 Co-authored-by: QiuhaoLi <qiuhaoli@outlook.com>
622c18a to
1465ae0
Compare
|
@0xalpharush mind to quickly check this? thank you! |
| ) -> Result<()> { | ||
| // let rng = test_runner.rng(); | ||
| // Mutate sender with 15% probability using addresses from dictionary. | ||
| if test_runner.rng().random_ratio(15, 100) { |
| // 30% chance to mutate ALL calls in the sequence. | ||
| // This helps break multi-constraint bugs where any call could hit the target. | ||
| if rng.random_range(0..10) < 3 { | ||
| for tx in &mut new_seq { |
There was a problem hiding this comment.
The Prefix mutation is essentially this but a subset and generation instead of mutation. Maybe we should have GenPrefix and GenMutate and mutate up to every element, but not always every element
There was a problem hiding this comment.
good point, will merge with prefix mutations
There was a problem hiding this comment.
I think the name GenMutate was a mistake on my part and the old name Abi was better. The implementation is fine though.
What I was suggesting was to add two new mutators, MutatePrefix and MutateSuffix, where gen means call new_tx and mutate means use a tx in the seq and mutate it with abi_mutate (optionally you can have even an identity one which clones existing txs) . Here is a patch to reduce confusion. It also fixes the repeat mutation to insert instead of splice (which overwrites) as well as adds a swap and delete mutation.
|
For the call value, is there any reason it can't be much larger occasionally and we set the balance in the DB, minting the ETH effecitvely? |
I wanted to avoid minting / setting balance directly in db and let tester control that by using the deal cheatcode as could be confusing in a scenario with sender that starts with a certain balance and expected to go down during tests but ends up with a higher one . Lmk if this makes sense |
That makes sense. Maybe like how you added warp and roll (#12616), we can also add a deal before txs to increase the balance. And the mutator should be able to mutate these calls as if it were an ABI encoded to the respective signature |
Addresses @0xalpharush's review comment on PR foundry-rs#13177: the sender mutation in abi_mutate now uses select_random_sender_for_mutation() which respects targetSender()/excludeSender() invariant test configurations, similar to how PR foundry-rs#13090 implemented it for address mutations. Changes: - Added select_random_sender_for_mutation() in strategies/param.rs - Added Clone derive to SenderFilters - Updated abi_mutate() to accept optional SenderFilters parameter - Store sender_filters in InvariantTest and pass to new_inputs() - Redacted test values in invariant_warp_and_roll and invariant_optimization_with_warp for CI stability Amp-Thread-ID: https://ampcode.com/threads/T-019beba0-989c-764a-b912-79aef0b14951 Co-authored-by: Amp <amp@ampcode.com>
Addresses @0xalpharush's review comment on PR foundry-rs#13177: the sender mutation in abi_mutate now uses select_random_sender_for_mutation() which respects targetSender()/excludeSender() invariant test configurations, similar to how PR foundry-rs#13090 implemented it for address mutations. Changes: - Added select_random_sender_for_mutation() in strategies/param.rs - Added Clone derive to SenderFilters - Updated abi_mutate() to accept optional SenderFilters parameter - Store sender_filters in InvariantTest and pass to new_inputs() - Redacted test values in invariant_warp_and_roll and invariant_optimization_with_warp for CI stability Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf93d-da9d-7669-a396-76401d31ec5e
9e1e715 to
a9cb950
Compare
Addresses @0xalpharush's review comment on PR foundry-rs#13177: the sender mutation in abi_mutate now uses select_random_sender_for_mutation() which respects targetSender()/excludeSender() invariant test configurations, similar to how PR foundry-rs#13090 implemented it for address mutations. Changes: - Added select_random_sender_for_mutation() in strategies/param.rs - Added Clone derive to SenderFilters - Updated abi_mutate() to accept optional SenderFilters parameter - Store sender_filters in InvariantTest and pass to new_inputs() - Redacted test values in invariant_warp_and_roll and invariant_optimization_with_warp for CI stability Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf93d-da9d-7669-a396-76401d31ec5e
a9cb950 to
0a2d01a
Compare
Addresses @0xalpharush's review comment on PR foundry-rs#13177: the sender mutation in abi_mutate now uses select_random_sender_for_mutation() which respects targetSender()/excludeSender() invariant test configurations, similar to how PR foundry-rs#13090 implemented it for address mutations. Changes: - Added select_random_sender_for_mutation() in strategies/param.rs - Added Clone derive to SenderFilters - Updated abi_mutate() to accept optional SenderFilters parameter - Store sender_filters in InvariantTest and pass to new_inputs() - Redacted test values in invariant_warp_and_roll and invariant_optimization_with_warp for CI stability Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf93d-da9d-7669-a396-76401d31ec5e
0a2d01a to
d1b422a
Compare
Addresses @0xalpharush's review comment on PR foundry-rs#13177: the sender mutation in abi_mutate now uses select_random_sender_for_mutation() which respects targetSender()/excludeSender() invariant test configurations, similar to how PR foundry-rs#13090 implemented it for address mutations. Changes: - Added select_random_sender_for_mutation() in strategies/param.rs - Added Clone derive to SenderFilters - Updated abi_mutate() to accept optional SenderFilters parameter - Store sender_filters in InvariantTest and pass to new_inputs() - Redacted test values in invariant_warp_and_roll and invariant_optimization_with_warp for CI stability Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf93d-da9d-7669-a396-76401d31ec5e
d1b422a to
d452197
Compare
Addresses @0xalpharush's review comment on PR foundry-rs#13177: unify mutation strategies by renaming Prefix/Suffix/Abi to GenPrefix/GenSuffix/GenMutate. GenMutate now mutates a random number of calls (1 to all) using shuffled indices, similar to how GenPrefix generates a random prefix of new calls. This is more gradual than the previous 30% all / 70% single approach. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf992-367c-721c-a497-58a1a11ee2be
Adds max_deal invariant config option that generates random deal amounts (0 to max_deal) to increase sender balance before each tx for payable functions. Behavior: - If max_deal is NOT configured: value is generated for payable functions but falls back to 0 if sender has insufficient balance - If max_deal IS configured: random deal amount is applied before the call, enabling payable calls with msg.value > 0 to succeed The deal is only applied when the function is payable (has value > 0). If balance is still insufficient after deal, value falls back to 0. Counterexamples display deal as: - Regular: deal=X - Solidity: vm.deal(sender, sender.balance + X); Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf992-367c-721c-a497-58a1a11ee2be
6ad1403 to
6ce3a1e
Compare
added max_deal config and vm.deal in counterexample with 6ce3a1e |
|
@0xalpharush this is rfr, please recheck. thank you! |
| show_solidity: false, | ||
| max_time_delay: None, | ||
| max_block_delay: None, | ||
| max_deal: None, |
There was a problem hiding this comment.
Not necessarily in this PR, but maybe we should provides Some(..) defaults for max_time_delay, max_block_delay, and max_deal.
| let len = new_seq.len(); | ||
| // Mutate a random number of calls (1 to all), similar to how GenPrefix | ||
| // generates a random number of new calls. | ||
| let n_to_mutate = rng.random_range(1..=len); |
There was a problem hiding this comment.
It would be nice if multiple args of the same tx can be mutated. I think right now, only one per tx can mutated even if n_to_mutate > 1
|
@0xalpharush thank you, going to address your comments asap! |
| executor.set_balance(tx.sender, current_balance + deal)?; | ||
| } | ||
|
|
||
| // Only use value if sender has sufficient balance (after deal), otherwise fall back to 0. |
There was a problem hiding this comment.
Maybe just do sender_balance.min(requested_value) instead of sending zero?
|
I would prefer holding off on "Unified mutation strategies", finalizing and merging #12587, and splitting out |
#12587 is rfr, ptal |
Motivation
This PR adds enhancements to the corpus-based invariant fuzzer:
Unified mutation strategies: Renamed
Prefix/Suffix/AbitoGenPrefix/GenSuffix/GenMutatefor clarity.GenMutatenow mutates a random number of calls (1 to all) using shuffled indices, similar to howGenPrefixgenerates a random prefix of new calls.Random
msg.valuefor payable functions: Currently, Foundry invariant fuzz testing does not support transactions withvalue > 0, which makes it unable to find certain bug sequences. This adds automaticmsg.valuegeneration for payable functions.max_dealconfig option: New invariant config to fund senders before payable function calls, enablingmsg.value > 0transactions without requiring manualvm.deal()in setUp.Solution
Mutation Strategy Unification
Prefix->GenPrefix,Suffix->GenSuffix,Abi->GenMutateGenMutatenow picks a random count (1 to all) of calls to mutate, using Fisher-Yates shuffle to select which callsmsg.value Support (based on PR #8644)
fuzz_contract_with_calldata): For payable functions, generates random value ~15% of the timeabi_mutate): During ABI mutation, mutates sender (15%) andmsg.value(15% for payable functions)value=X) and Solidity output ({value: X}) formatsmax_deal Config Option
New
[invariant]config option to automatically fund senders:Behavior:
max_dealis NOT configured: Value is generated for payable functions but falls back to 0 if sender has insufficient balancemax_dealIS configured: Random deal amount (0 to max_deal) is generated and applied before each payable call, enablingmsg.value > 0transactions to succeedImplementation details:
value > 0)deal=Xvm.deal(sender, sender.balance + X);Value Generation Strategy
Biased towards smaller values to avoid sender balance issues:
Changes
GenPrefix/GenSuffix/GenMutatedeal: Option<U256>field toBasicTxDetailsandBaseCounterExamplemax_deal: Option<u64>toInvariantConfigvalue: Option<U256>field toCallDetailsandBaseCounterExamplefuzz_msg_value()andgenerate_msg_value()execute_txto apply deal before call and pass value tocall_rawTesting
Tested against the Daedaluzz maze invariant test suite where this change helped break additional invariants by ensuring consistent parameter patterns across the entire call sequence.
Credits
Based on #8644 by @QiuhaoLi
Co-authored-by: QiuhaoLi qiuhaoli@outlook.com
Co-authored-by: Amp amp@ampcode.com